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Analysis  of  Results 


■  Cost  effectiveness  of  defensive  investments  has  a  major 
effect  on  the  optimal  resource  allocation 

■  When  investment  is  not  highly  cost  effective: 

■  All  or  most  of  the  budget  should  go  to  most  attractive  target(s) 

■  As  the  cost  effectiveness  increases: 

■  Smaller  targets  get  more  funding 

■  But  the  most  attractive  target  still  gets  a  larger  share 

■  Different  measures  of  attractiveness  yield  different 
optimal  budget  allocations 
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Motivation 


Cost  effectiveness  of  defensive  investment  has  an 
enormous  impact  on  optimal  allocation  of  defenses: 

-  But  we  do  not  yet  have  good  estimates  of  cost  effectiveness 

I  will  present  quantitative  estimates  of  the  cost- 
effectiveness  of  investments  in  protection  and  resilience 

-  Based  on  observed  reductions  in  estimated  criticality  after  the 
expenditure  of  security  funds 


Data 


Wisconsin  Office  of  Justice  Assistance  (OJA)  provided: 

-  A  sanitized  list  of  critical  infrastructures  and  key  resources 

-  The  dollar  amount  spent  by  each  site  (from  $0  to  $485,000) 

-  Each  site’s  before  and  after  criticality  scores  (from  36  to  56,  on 
scale  of  0  to  100) 

Data  included  assets  in  the  following  sectors: 

-  Hazardous  Materials 

-  Water 

-  Commercial 

-  Transportation 

-  Government 


Criticality  Scores 


Criticality  scores  were  developed  using  the  Critical  Asset 
Risk  Evaluation  System  (CARES)  developed  by  IEM. 

CARES  is  an  automated  risk-assessment  tool  that  helps 
users  analyze  and  compare  relative  infrastructure  risks, 
using  the  basic  DHS  risk-management  methodology: 

-  RISK  =  THREAT  *  VULNERABILITY  x  CONSEQUENCE 


Criticality  Scores 


THREA  T 

-  Threat  Indicators 

-  Threat  History 

VULNERABILITY 

-  Access  Denial 

-  Threat  Detection 

-  Incident  Termination 


Criticality  Scores 

CONSEQUENCE 

-  Death  and  Injury 

-  Public  Health,  Safety,  and  Security 

-  Economic  Impact 

-  Government  Operations 

-  Psychological  Influence,  Public  Confidence, 
and  Morale 

-  Destruction  of  Property 

-  Environment  Impact 

-  Impact  on  Additional  Critical  Infrastructure 


Statistical  Analysis 

Dependent  Variable: 

-  Risk  Reduction  (RR)  =  1  -  Final  Score  (F)/Original  Score  (O) 

Statistically  Significant  Independent  Variables: 

-  Intercept 

-  S:  Amount  Spent  (in  thousands  of  dollars) 

-  O:  Original  Criticality  Score 

-  T:  Transportation  Sector  (binary  variable) 


Fitted  Regression  Model 

Ln(RR)  =  -  3.75  -  0.001 9  S  +  0.0395  O  -  1 .04  T 

-  Std.  error:  (0.68)  (0.0004)  (0.0140)  (0.20) 

RR  =  0.023  (0.998s)  (1 .04°)  (0.357) 

Adjusted  R2  =  0.80 
Example: 

-  If  the  original  criticality  score  (O)  is  50 

-  The  amount  spent  is  $100,000  (S  =  100) 

-  The  asset  is  not  transportation  (7=0) 

Then  the  risk  reduction  is  estimated  to  be: 

-  (0.023)  (0.998100)  (1.0450)  (0.35°)  = 

-  0.023  (0.82)  (7.11)  (1)  =  0.13 


Fitted  Regression  Model 


Reasonable  fits  were  achieved: 
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Results  were  also  quite  robust  with  model  formulation: 

-  E  g.,  Ln(RR)  vs.  RR,  Ln(O)  vs.  O,  Ln(RR)  vs.  Ln(l-RR) 


Findings 

For  every  $100K  spent  (all  else  equal),  17%  less  risk 
reduction  was  achieved! 

-  This  does  not  imply  that  spending  more  money  increases  risk 

-  Only  that  there  is  a  wide  variation  in  cost-effectiveness  of 
investments  between  sites 


Findings 


For  every  10-point  increase  in  the  original  criticality  score 
(all  else  equal),  50%  greater  risk  reduction  is  achieved 

-  In  other  words,  sites  with  higher  original  risk  tended  to  have  more 
cost-effective  improvements 

-  “Low-hanging  fruit” 

So$100K,  T*C 


Findings 

The  two  transportation  sites  were  significantly  less  cost- 
effective  than  sites  in  the  other  sectors: 

-  65%  less  reduction  in  risk,  all  else  equal 

However,  this  observation  should  be  treated  with  care: 

-  Since  there  were  only  two  transportation  sites  in  the  data  set 


Future  Work 


We  have  enough  experience  by  now  with  methods  like 
TRAM  to  generate  more  complete  and  reliable  data  sets 

What  is  the  next  step  in  generating  order-of-magnitude 
estimates  of  cost  effectiveness  for  defense? 
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